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Foreword 


Cedefop has been at the forefront of developing robust skills anticipation 
methods and skills intelligence tools for the European Union for more 
than a decade. The European skills forecast and the European skills and 
jobs survey shed light on how the labour market, skill needs and jobs are 
developing and help signal potential skills bottlenecks. Cedefop’s big data 
analysis of online job advertisements provides detailed and real-time skills 
intelligence, capturing which skills have currency in job markets. Cedefop 
has used skills foresight to develop stakeholder-backed policy roadmaps 
aimed at strengthening national skills anticipation and matching systems. 
Complementing quantitative skills analysis and intelligence, qualitative 
insight into skills policies and measures also contributes to evidence-based 
policy-making. 

The continuing development of national skills intelligence systems 
and approaches has helped strengthen the feedback loops between 
the labour market and vocational education and training (VET) and skills 
policy. In the coming years, we need to be more ambitious. Our vision for 
Skills intelligence 2.0 is information that is more actionable: detailed and 
relevant, better contextualised, timelier, and better communicated. Making 
sense of trends and fostering capacity to act on them means combining 
sources and approaches - skill surveys, skills forecasting, skill foresight, 
big data analyses, and others — and exploring synergies. This gives policy- 
makers the means to separate noise from signal and supports employers 
and citizens in making decisions in line with the new realities in the world 
of work. 

It is no surprise that skills intelligence is a key priority in the 2020 European 
skills agenda. Reliable and fit-for- purpose labour market and skills intelligence 
has enormous value in times of rapid change and transformation. In a context 
of fast-paced digital advancements, such as artificial intelligence and 
advanced robotics, and other megatrends such as population ageing and 
the green transition, VET and skills policies should become more proactive. 
To prepare new generations of learners and to support people in making and 
shaping career transitions, reliable skills intelligence is indispensable. 


OO oO 


PREVIOUS CONTENTS NEXT 


Foreword 


This publication is the first in a series of practical skills anticipation 
guides for policy-makers and analysts. The guides present a rich mosaic of 
conventional and emerging methods for identifying technological change and 
its impact on skills. Systematically presenting the merits and disadvantages 
of different methods, they show no single approach can provide all the 
answers. Apart from reliable data and sound methods, creativity, holistic 
thinking and using collective wisdom to shape the future are key building 
blocks of skills intelligence 2.0. 

This first guide focuses on conventional methods of anticipating changing 
technologies and skill demands: skills surveys and skills forecasts. We trust 
the practical insights it provides will prove to be useful in your context. 


Jiirgen Siebel Antonio Ranieri 
Executive Director Ad interim head of department 
for skills and labour market 
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CHAPTER 1. 
Technological change 
and skills 


14.1. The fourth industrial revolution 


Popular media and the vocational education and training (VET) and skills policy 
discourse highlight that technological change taking place in European firms 
is prevalent and fast-paced. The impression is that the world of work is at 
a crossroads of a fourth industrial revolution, being transformed by Industry 4.0, 
advanced robotics, artificial intelligence (Al), the internet of things (loT) and other 
emerging technologies in a way that is more profound than previous waves of 
change, such as the one driven by the microprocessor revolution of the 1970s. 
This may or may not be true. But with forecasts suggesting that large shares of 
the current workforce are at risk of having their jobs substituted or transformed 
by automation (Frey and Osborne, 2013; Arntz et al., 2017; Nedelkoska and 
Quintini, 2018; Pouliakas, 2018), the pressure is on those responsible for skills 
anticipation to come up with answers to several important questions. 
Some of these are concerned with potential threats: 
(a) which technologies are likely to emerge and become widely used over the 
short to medium term? 
(b) what jobs will they impact? 
(c) which skill sets are likely to become obsolete? 


Others focus on opportunities: 
(a) what new tasks and jobs will result from technological change? 
(b) what skill sets do people need to acquire to fill those jobs? 
(c) how does VET need to adapt? 


1.2. Identifying technological change 


Identifying and anticipating the pace of technological change in labour 
markets is challenging; an example is the early estimates of about half of 
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all jobs in developed economies being at potential risk of substitution by 
machine learning algorithms (Frey and Osborne, 2013; 2017). These findings 
have been widely dismissed in several follow-up studies, which adopted 
alternative methodological approaches. Later estimates of the share of jobs 
disappearing due to automation based on a task or skills-based approach 
include using Cedefop’s European skills and jobs survey and the OECD’s 
(Organisation for Economic Cooperation and Development) survey of adult 
skills (programme for the international assessment of adult competences, 
PIAAC); these have been much lower than the early worryingly high 
predictions. Analysts have also looked at indirect indicators, such as total 
factor productivity, or direct measures of technological change. Such 
measures include the level of computer investment in an industry, the 
percentage of firms introducing particular technology (such as robotics or 
Al technologies) and the share of workers using new computer software 
or equipment (McGuinness et al., 2019). While all of these measures 
provide useful insights into how technological progress is affecting skill 
needs in labour markets, their measured effects can be piecemeal and the 
assumptions underlying them are sometimes challenged. 


1.3. Understanding and measuring skills trends 


To understand the extent to which technology is transforming the world of 
work, it is necessary to measure its magnitude and impact on skills demand. 
Labour market and skills intelligence (LMSI, often referred to as skills 
intelligence) provides such information and — provided that it is based on 
sound approaches and methods - can serve the needs of those responsible 
for reacting to changing skill needs. Such LMSI needs to provide information 
on the current situation and also reveal something about the future. Policy- 
makers are likely to have more opportunities to influence the future rather 
than the immediate skill supply. And if they are to make decisions that have 
some bearing on the behaviour of labour market actors, then they need to 
have sufficient level of detail. 

Telling policy-makers and relevant stakeholders that the demand for 
people employed in professional occupations is likely to increase may not 
be of much value for them in making decisions about skills or qualifications 
requiring investment. They need to know job details and the specific skills 
attached to it. Similarly, if the concern is about the risk of automation wiping 
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out jobs, there is a need to know which jobs. Simply saying that people 
employed in the broad plant and machine operator occupational group — as 
defined by the international standard classification of occupations (ISCO) — 
are more likely to engage in routine tasks, and hence face higher automation 
risk, is of limited value to a decision-maker. There is a need to know which 
specific jobs within that occupational category are at risk (Freeman et al., 
2020). 

It is often the case that the tasks and skills within a job need to be the 
focus of attention, especially the degree to which they can be rendered 
obsolete or new ones created (Deming and Noray, 2020). It is often not so 
much the job which is at risk, but rather particular tasks and skills required to 
carry out that job. By focusing on the latter, there may be a better chance to 
offset the risk that automation might pose to some jobs. This will be feasible 
by equipping affected individuals with the skills that the reconfiguration of 
work implies following the adoption of a given technology. 

But what is meant by skill? It is a word or concept which is freely used 
without much certainty that it means the same thing to different people or 
labour-market actors. Before considering how data on current and future 
skill demand related to technological change can be collated, it is necessary 
to agree on common use of terms. 

A skill is any capability that satisfies some practical requirement of 
a job but it is an elusive concept to define (Attewell, 1990). Economists’ 
definitions of it have been largely driven by an interest in estimating the 
returns to human capital investments (Becker, 1962; Mincer, 1974). Skill is 
largely defined with reference to its capacity to deliver a financial return; 
a higher return suggests a higher skill level. By contrast, within psychology, 
the concept of skills typically refers to the qualities workers possess and 
the capabilities required for accomplishing certain work tasks, often referred 
to as competence. Sociologists have also been mostly concerned with the 
ability of individuals to respond to work complexity and the value of skills as 
part of social processes. 

The skills required by jobs are diverse and multidimensional, and they 
can be specified in potentially infinite levels of detail. No survey or study 
can capture all skills involved in a particular job because any description of 
what a job entails can always be enriched with further detail. There is also 
a tension between detail and comparability across occupations: very detailed 
measures tend to be occupation-specific, while overly general measures risk 
being weakly informative. The key is to measure transversal skills and to 
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devise measures pitched at a mid-level of generality that are relevant across 
a range of occupations. It should also include a reasonably concise checklist 
of more specific requirements, such as an inventory of digital skills, which 
are particularly relevant for research and policy. Most of the interest in skills 
and job-skill requirements typically focuses on a division of the concept 
at a most general level into cognitive, interpersonal, and manual skills. 
Interpersonal or ‘soft’ skills have proven to be weakly conceptualised, as 
they also often include more purely attitudinal and motivational aspects of 
work orientations (Moss and Tilly, 2001). By contrast, cognitive and manual 
skills tend to be more concisely measured and are usually associated with 
robust labour market outcomes for individuals. 

In econometric equations, skill has been often defined with reference to 
occupation and/or qualifications which are, at best, proxy measures of the 
phenomenon. Because data on occupations and qualifications -— measured 
at international level by ISCO and the international standard classification of 
education (ISCED) — are readily available, they are typically used as measures 
of skill in economic studies. This holds even though there is no guarantee 
that the skills acquired in obtaining a qualification are actually used by the 
individual in their job, giving rise to skill mismatches (McGuinness et al., 2018). 

Given the difficulty in clearly defining the meaning of skill, analysts and 
labour economists have embraced the concept of job-skill requirements 
(Handel, 2012). Job-skill requirements, in short, refer to the specific tasks 
which comprise a job and the skills needed to undertake those tasks 
proficiently. With the emergence of surveys that have collected data on job- 
skill requirements, more detailed understanding has been obtained of the 
tasks which comprise a particular job and the skills which are needed to 
carry out those tasks successfully (Green, 2013). The job-skill requirements 
approach was pioneered in the UK’s skills and employment surveys, 
which have been periodically carried out since 1986; it and is central to 
the OECD’s survey of adult skills (PIAAC) and the skills, technology and 
management practices survey (STAMP) conducted in the USA (Handel, 
2016; 2017). To give an idea of the type of information collected using the 
job-skill requirements approach, Table 1 summarises the skills information 
collected by the STAMP survey of workers. 

The importance of the job-skill requirements approach is that it provides 
the basis for better understanding the relationship between skills and 
technology, including the risk of skills obsolescence and job loss resulting 
from technological change. There is a strong focus in the literature on 
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identifying jobs or occupations that comprise a range of work activities that 
are more or less susceptible to being replaced by automation. The work of 
Autor and colleagues (2003) is groundbreaking in that it appears to have 
made the first use of the US Dictionary of occupational titles (DOT) to provide 
information about the nature of the tasks performed within occupations. The 
analysis suggested a hollowing out of the labour market, given that routine 
jobs — mainly those located in the middle of the occupational hierarchy 
(skilled trades and machine and assembly workers) — can be more easily 
substituted by machines, than jobs at the lower and higher ends of the jobs 
spectrum. 

More recent analysis, such as that based on the Cedefop European skills 
and jobs survey (ESJS), suggests that automation can affect some tasks 
within a job, but not necessarily every task (Pouliakas, 2018). This means 
analysis must look in detail at intra-occupational task change resulting from 
technological change. The key point here is that, in understanding the nature 
of the relationship between skills and technological change, one needs to 
be precise in identifying skills affected by technological change, rather than 
solely focusing on whole jobs or occupations. 


Table 1. Skill measures used in the skills, technology and management 
practices survey 


Skill and task requirements 

e Cognitive skills 
(maths/numeracy, reading, writing, 
problem-solving) 

e Interpersonal skills 
(including teamworking) 

¢ Physical job demands/physical skills 


Computer use (digital skills) 

e Computers 

e Frequency of use 

e Use of 14 specific applications 

e Use of advanced programme features 

e Job-specific and new software 

e Training times 

¢ Complexity and computer skills required 

e Adequacy of respondents’ computer skills 

¢ Computer experience of non-users in 
previous jobs 


Source: Handel, 2017. 


Machinery and electronic equipment 

e Set up maintenance and repair 

e Equipment and tool programming 

© Mechanical and electronics knowledge 


Supervision, autonomy and authority 

© Closeness of supervision, autonomy and 
repetitiveness 

e Supervisory responsibilities 


General measures of skill 

e Required education for job 

e Required experience 

e Employer-provided training 

e On-the-job training 

e Training in specific skills 
(literacy, math, customer service, sales, 
managerial skills, communication, quality 
control, technical skills) 
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1.4. Collating skills intelligence 


Provided that an analyst has a good grasp of which skills to measure and how, 
what data collection approach offers insight into the impact of technological 
change on them? Something is required that delivers detail and, in an ideal 
world, insights into future skill needs. This is not straightforward and, to 
date, no single approach to skills anticipation has managed to reconcile 
fully the provision of both detail and a future-looking perspective. However, 
improvements are continuously being made and many skills forecast 
approaches are pushing the boundaries with respect to the amount of 
occupational detail that can be provided. 

But there are problems, too. Where data are often provided on the current 
situation of a labour market, it is often the case that information is already 
dated because of the time lag between data collection and the publication 
of findings. This results from the rigour with which official statistics are 
collected, cleaned, and weighted before publication. The future is also 
becoming more volatile in some respects. The types of technology which are 
being introduced, and the speed at which this takes place, the way in which 
they transform work organisation and the types of skill needs this gives rise 
to, may be without historical precedence. 

The sophisticated extrapolation of time series data into the future, typically 
applied by economists, may be misplaced if the past becomes a less reliable 
guide to the future. This perspective is sometimes criticised as an exaggeration 
because technological innovation tends to become mainstream at a slower 
pace than some commentators suggest (Bessen, 2016). It can be argued that 
conventional methodologies and tools at hand to examine the relationship 
between technological change and skill demand remain robust and reliable. 

Table 2 summarises some of the main methods that can be used to gather 
information on skills needs. Four are particularly important: 

(a) ask questions to key stakeholders (questionnaire surveys of employers’ 
and employees’ skill needs and experience of technological change); 

(b) produce quantitative estimates of future skill demands, by extrapolating 
past trends and modelling expected developments; 

(c) source ‘big’ data on new technologies and skills from a variety of online sources 

(for example job portals, CVs, social media, patents, scientific databases); 
(d) use non-quantitative techniques, relying mostly on participatory stakeholder 

approaches to gauge in-depth information about the state of current and 

future skill demand and supply. 
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Table 2. Tools for carrying out skills assessment and anticipation 


Type of activity Data collected 


Descriptive statistics/ 
stock taking 


Estimates of overall demand and supply of skills 
and technology use, often based on collating data 
from various sources (for example, sector skill 
studies) 


Quantitative forecasting 


Forecasting or projecting future demand for skills 
typically using econometric modelling 


Skills and jobs surveys 
(questionnaire surveys) 


Graduate tracer studies 


Assessments of demand for, and supply of, skills 
and technology use, usually with an assessment 
of the extent to which demand and supply are in 
balance 


Using matched administrative datasets or surveys 
to track people through education and the labour 
market to see how the former influences the latter 


Qualitative research 


Use of non-quantitative techniques to gauge in- 
depth information about current and future skill 
demand/supply and technology trends, e.g. via 

company case studies, use of focus groups 


Foresight Critical thinking about the future of skills 
supply/demand and technology trends using 
participatory methodologies 

Big data Use of web sourcing combined with text mining and 


Source: Cedefop classification. 


machine learning approaches to collect and classify 
data about skills, vacancies, technologies 


1.5. Purpose of guides 


This first Cedefop ‘how-to’ guide on understanding the impact of technological 
change on skill demand, focuses on conventional approaches to skills 
anticipation that typically employ quantitative methods, notably econometric 
forecasting and survey-based analyses (such as employer and employee 


surveys). 


These are non-participatory approaches to skills anticipation in that 
they do not require stakeholders to be involved in deriving estimates of the 
impact of technological change on skills. There is no normative element as 
is present in participatory approaches, such as foresight, where the aim is 
sometimes to bring about a certain outcome, as covered in the third guide 
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of this series ('). The approach in this guide is on collecting representative 
data to produce robust and reliable findings, using scientific statistical 
approaches. 

This differentiates such methods from the increasing use of big data and 
Al-driven data analysis, which is a new method in the toolkit of analysts. 
While such big data approaches —- covered in Cedefop’s second guide (?) — 
are innovative, they cannot fully substitute for conventional statistical 
methods based on random probabilistic approaches. A guiding principle 
of all Cedefop guides, however, is that the best skills intelligence can be 
obtained by combining various types of analysis in a way that will reveal 
a holistic and robust view of emerging trends and their likely impact on skills 
demand in labour markets. 

This guide is structured as follows. Chapter 2 focuses first on the 
development and implementation of employer or worker skill surveys, as 
a means to extract information of the use and impact of technologies at 
work. Chapter 3 then turns to the use of more elaborate skills forecasting 
quantitative models and ways of modelling technology trends in them. 
Chapter 4 concludes with a discussion on which LMSI is best to choose and 
the reasons for doing so. 


(‘) Cedefop (2021a). Understanding technological change and skill needs: technology and skills 
foresight. Cedefop practical guide 3. Luxembourg: Publications Office. http://data.europa.eu/ 
doi/10.2801/307925 

(*) Cedefop (2021b). Understanding technological change and skill needs: big data and artificial 
intelligence methods. Cedefop practical guide 2. Luxembourg: Publications Office. 
http://data.europa.eu/doi/10.2801/144881 
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Box 1. Cedefop practical guides on understanding technological 


change and skills demand 


The purpose of Cedefop’s short ‘how-to’ guides is to provide those with a responsi- 
bility for undertaking skills assessment and anticipation with the means to deal with 
the uncertainty of technological change and its impact on skill needs. 

As the process of predicting the future becomes more complicated and less de- 
terministic, the range of tools available to those involved in skills anticipation has 
become more varied and sophisticated. The Cedefop guides aim to showcase to pol- 
icy-makers and interested analysts how various techniques or methodological tools 
can be readily applied, by carefully considering the associated pitfalls and rewards 
of doing so. 

The guides provide targeted information on how interested analysts can adopt and 
implement conventional labour market and skills intelligence methods, such as 
skills surveys and skill forecasts; automated methods reliant on big data and artifi- 
cial intelligence (Al) techniques; or technology foresight methods. All can be used to 
detect emerging skill needs related to technological change. Implicit in the guides 
is recognition that no one methodology is likely to provide all the answers and the 
Challenge for analysts is to bring together outputs from different approaches to 
skills anticipation. 

The guides build on the existing compendium of guides on skills anticipation pro- 
duced by the ETF, Cedefop and the ILO (3), as well as several previous Cedefop 
reports on skills anticipation methods (*). But they are distinct from previously pub- 
lished methodological handbooks or guides, in that they are explicitly concerned 
with the process of identifying technological (digital) change, a key driver of chang- 
ing skill needs. 


Source: Cedefop. 


() 


https://www.cedefop.europa.eu/en/news-and-press/news/first-comprehensive-compendium- 
guides-skills-anticipation-methods 

For instance, see ETF et al. (2013), Cedefop (2015) and Cedefop’s project Anticipating and 
matching skills. 
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CHAPTER 2. 

The art of asking questions: 
employer and employee skills 
surveys 


Employer and employee skill surveys can be used to collect data on the 
introduction of technological change and how it has impacted on skills 
demand among a representative cross-section (or panel) of companies, 
workplaces or workers. It is often more useful and practical to conduct 
a survey of workplaces rather than multi-site companies, because 
respondents are typically better able to answer questions about their direct 
work environment, compared to questions about the bigger company 
that their workplace may be part of. While typically referred to as worker 
surveys, surveys among individuals can also be among economically active 
individuals in a wider sense. 


2.1. Employer skills surveys 


Employer surveys can provide relevant information on the following aspects 

of technological change and its implications for skill demand: 

(a) prevalence of technological change (for example, the extent to which an 
establishment has introduced new machinery, equipment or technology 
over the recent past for reasons other than routine replacement) and 
plans to introduce technology in the future; 

(b) vacancies and the extent to which they are hard to fill because applicants 
lack the skills, qualifications or experience required. Such information can 
typically be disaggregated by occupational group; 

(c) internal skill gaps (for example the extent to which current staff is fully 
proficient at their jobs, again this can be disaggregated by occupational 
group); 

(d) provision of workforce training. 
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Potentially this allows for analysis of the relationship between investments 
in various kinds of technological equipment and levels of skill shortage 
vacancies and internal skill gaps. If the data can be disaggregated by 
occupation, it is possible to gain insight into the extent to which technological 
change affects skill demand and whether that skill demand can be met by 
either the company’s existing workforce or via external recruitment. 

There are, however, some caveats: the capacity of the employer to report 
correctly the extent of technological change; and the tendency of some 
employers to overstate skill shortages because it may be in their interest to do 
so, if they want to see the supply of certain skills increased (Gambin et al., 2016). 

Asking employers about the future usually produces mixed results. While 
some employers have a good overview of the current and future challenges 
they face, others are less prepared and are likely to consider the future only 
at the point at which it arrives. A Dutch sectoral study, where firms were 
surveyed on current and future employment developments, clearly illustrates 
this. In some cases, the average predicted future demand coincided with 
reality one quarter later but this was by no means the case in many other 
instances (Fouarge et al., 2012). In a heterogeneous population of employers, 
some will be using cutting-edge methods to predict future employment, 
while others will be doing next to little skills assessment, if any at all. 

There is also a risk that planned technological change is under-reported. 
The German vacancy survey, a representative establishment survey with 
more than 10 000 responses, included, in its 2016 round, aseries of questions 
on anticipated changes relating to digitalisation over the next five years. 
More than half of the respondents said that they did not expect digitalisation 
(internal digital networking, networking with customers/suppliers, or the 
use of learning systems) to increase over the next five years. If true, this 
is a remarkably low share of employers affected by technological change 
(Warning and Weber 2017; 2018). Perhaps some employers simply do not 
think that far ahead. While digitalisation trends can be identified to some 
degree, the timing and the speed of change are hard to measure. Repeated 
(similar) surveys among similar (or the same) firms, carried out in order to 
pinpoint the speed and convergence of new technologies, trends and skills 
required within organisations, can help overcome such problems. 

Employer skill surveys are not particularly suitable for collecting 
information on how the content of jobs (occupations) in the workplace 
is changing, and the extent to which changes have emerged because of 
technological change of some kind. Company respondents (chief executive 
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or personnel manager) find it difficult to answer questions about different 
categories of workers, especially if it is a large workplace. Cedefop’s pilot 
employer survey on skill needs (Cedefop, 2013) made some headway in 
this regard, by collecting data on the importance of various transversal 
and occupation-specific skills in selected occupations and whether or not 
this importance has changed over time. In this way the survey was able 
to incorporate a skill requirement type approach. But widely implementing 
a detailed occupation-based employer survey can be costly. It may also 
be challenging to overcome serious methodological constraints, related 
to the sampling registers capturing the occupational distribution within 
companies or establishments across different countries (Cedefop, 2013). 

The 2019 fourth European company survey (Eurofound and Cedefop, 
2020) contains information on technology use (including use of computers 
and software, robots, use of data analytics), the extent to which employees’ 
skills are matched to their job, and the pace at which the job content changes. 
Based on the questions asked, it is possible to gain insight, other things 
being equal, into the relationship between technology use and skill needs 
and mismatches in the workplace. However, such information is collected at 
an aggregate establishment level, giving rise to potential ecological fallacies 
that occur when inferences are made about individuals drawn from groups 
that they may belong to. 

Table 3 provides a summary of the issues to consider when thinking about 
using an employer survey to explore the link between technological change 
and skills. It is effectively a checklist of what needs to be considered before 
commissioning such a survey. 


Table 3. Issues to consider before launching an employer skills survey 


issues Detail to be considered 


Initial check Do similar data already exist elsewhere? 
Types of questions = Introduction of technological change over the recent past 
that can be asked (e.g. last 12 months)? 


about technologies © Adoption of specific technologies (e.g. computers, robots, Al)? 


Types of questions © Occupational structure of the workforce 
that can be asked (e.g. at ISCO one or two-digit level) 
about skills e Typical qualification level required to get a job 
in each occupation 
e Vacancies and recruitment difficulties (disaggregated by occupation) 
and reasons for difficult-to-fill vacancies 
e Extent of internal skill gaps (disaggregated by occupation) 
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fissues =| Detail to be considered 


Survey ¢ Sampling frame from which to derive a representative sample of the 
requirements population of employers of interest 
¢ Need to assess whether a company survey or establishment-based 
one is required. Establishment-based surveys are easier because 
the respondents tend to find it easier to answer questions about the 
workplace at which they are based 


Data collection ¢ Specifying hypotheses to be addressed 
steps © Specifying the variables and their relationships required to test the 
hypotheses 
e Drafting questions that will allow variables to be constructed (the 
questionnaire) 


e Crosschecking questions with other surveys 
(to allow for comparisons) 

© Design of sample (random probability, quota sampling, etc.) and sample 
size required for robust estimates to be derived 

¢ Type of survey (face-to-face, telephone, online, etc.) 

e Fieldwork 

e Weighting dataset to population estimates by employer and by 
employees employed in the workplaces sampled 


Selected further e ETF, Cedefop and ILO Developing and running an establishment skills 
literature survey 
Cedefop User guide to developing an employer survey on skill needs 
¢ Cedefop and Eurofound: 
European company survey 


Source: Cedefop 


2.2. Employee skills and jobs surveys 


Employee surveys can be more successful in capturing information about 
technological change and changes in the skill content of a job (especially 
various transversal skills, such as numeracy, literacy, digital skills), because 
they target individual workers directly impacted by it. The example of the 
STAMP approach provided in Table 1, which has also been used in other 
recent international surveys, such as the OECD’s survey of adult skills, the 
World Bank’s Skills towards employment and productivity (STEP) survey and 
Cedefop’s European skills and jobs survey, demonstrates how the concept 
of skill can be operationalised for data collection purposes. 

There is the possibility that employees may exaggerate their skill or 
technology use levels, borne out of a reluctance to acknowledge that they 
are at a lower level required by their job or at a low level overall. Nevertheless, 
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it can be beneficial to undertake an employee survey because of the 

possibilities it offers to collect information about: 

(a) the employee’s use (and level of use) of technologies at work and their 
experience of technological change in the workplace over a given time 
period; 

(b) the employee’s perception of how well their skills are matched to their 
current job; 

(c) the employee’s rating or assessment of their skill level, according to 
cognitive and other skill batteries or tests; 

(d) the importance of various skills to an individual’s job (for example 
how often they use those skills) and the level at which specific skills are 
held; 

(e) how the skill or task content of their job has changed over time. 


With respect to skills in particular, questions tend to be asked on: 

(a) required skill level; 

(b) whether the respondent possesses the requisite level of skill or exceeds 
it; 

(c) the frequency of skills use; 

(d) the importance of skills to the job; 

(e) the level of complexity of skills in the job; 

(f) how importance, frequency of use, and level of skill or complexity have 
changed over time. 


2.2.1. Job-skill requirements approach 

In the past, a lot of skills analysis relied upon using occupation and 
qualification as proxy measures of skill. These are imperfect measures. 
Broad occupation or education categories are coarse, roughly ordinal at 
best, and do not provide information on which skills are increasingly affected 
by technological change. Early employee skill surveys asked workers directly 
about their self-perceived level of skills, their importance for doing their job 
or how well-suited they are to fulfilling job requirements (see Cedefop’s first 
European skills and jobs survey). The OECD’s survey of adult skills (PIAAC) 
devised a more objective approach to assessing the skill level of individuals 
for three foundation skills: 

(a) literacy; 

(b) numeracy; 

(c) problem-solving in technology-rich environments. 
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To overcome the drawbacks associated with previous self-reporting 
approaches, such as lack of objective anchors/yardsticks leading to personal 
interpretations by different (groups of) respondents, recent skills surveys 
have increasingly adopted the job-skill requirements methodology (Handel, 
2017). This aids careful definition and operationalisation of the concept of 
skill needs, and has been a valuable development in the field. The job-skills 
requirements approach asks survey respondents about the specific tasks 
which comprise their jobs; these are usually mapped to specific skills, for 
instance cognitive, interpersonal, manual or digital skills, required to carry 
out the job successfully. Information on complexity of skills required in a job 
is typically obtained via incremental scale questions, which ask workers 
whether they undertake basic or more complex tasks in their job (Box 2). 


Box 2. Questions based on a job-skill requirements approach 


As part of your main job, do you regularly/did you do any of the following in the last 
(reference period): 
e Read any texts that are at least one page or longer/five pages or longer, etc.? 
> Literacy skills 
e Lift or carry heavy objects or loads? 
> Manual skills 
e Use basic mathematics, for instance addition, subtraction, multiplication or division? 
> Numeracy skills 
e Use any computing devices to send emails? 
> Digital skills 
e Etc. 


Source: Cedefop second European skills and jobs survey (ESJS) and STAMP (see Handel, 2017). 


The job-skill requirements approach uses explicit scaling, asking 
questions about workers’ regular activities carried out at work, translated 
into objective metrics (for example time or number of pages), rather than 
relying on abstract or vague questions on skills that are open to subjective 
interpretation and social desirability biases. Questions are general enough to 
encompass diverse work situations, but at the same time sufficiently concrete 
to have stable meanings across respondents. The job-skill requirements 
approach lends itself to surveys of individuals better than employers, simply 
because in an employer survey the respondent will inevitably be asked to 
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report on a wide range of jobs, some of which the respondent might not 


necessarily be familiar with. 


The objective, validated and standardised method of measuring job-skill 
demand administered consistently to representative samples of workers over 
time has aided understanding of the ways work is changing (Handel, 2003). 
The STAMP survey in the US (Handel, 2016), OECD’s PIAAC survey and 
Cedefop’s second European skills and jobs survey are notable examples of 
the job-skill requirements approach, which collects, in addition to information 
on the level of education required by the job and the duration of job-specific 
training, insights on the levels of cognitive, manual, interpersonal and ICT 


skills required in jobs. 
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Box 3. The job-skill requirements module of Cedefop’s second ESJS 


Building on the US STAMP survey, the second European skills and jobs survey col- 
lects, via a dedicated job-tasks module, information on the following job-skill re- 
quirements in EU workers’ jobs: 


Cognitive 

e Reading 

e Writing 

e Maths 

e Problem-solving 
e Creativity 


Manual 

e Lifting 

e Dexterity 

e Repetitiveness/ 
standardisation 

e Use of 
computerised 
machines 


Interpersonal 


¢ Counselling 

© Selling 

e Serving 

e Presenting 

e Teaching/ 
training 

e Persuading/ 
negotiating 

e Caring 

e Teamworking 


Digital 

e Email/internet/ 
social media 

e Word processing 

e Spreadsheets 

e Data management 

e Occupation-specific 
software 

e Programming (Al) 


It employs an incremental Guttman scaling approach that enables identification of 
higher or lower job-skill requirements in jobs. It also focuses on the discrete use of 
a specific task as part of an employee’s main job in a given (one-month) reference 


period. 


Source: Cedefop second European skills and jobs survey questionnaire. 


The application of the job-skill requirements approach in a survey requires 
overcoming several challenges during its design stage. Decisions need to be 
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made on how the survey will capture the regularity or periodicity whereby 
workers carry out some of their job tasks and if the focus should be on 
systemic or crucial job features, even if the latter are carried out occasionally. 
For example, it needs to be decided whether the focus should be on whether 
workers do a specific task regularly as part of their job, or frequently (a greater 
proportion of their working time) or if the task has been done at all within 
a given reference period? The dimension element is also important: how to 
strike a right balance between asking workers about a relatively broad or 
a detailed set of tasks? 

It is important to be aware that respondents’ answers may be affected 
by the inclusion or not of a reference period in the question. For some 
workers, tasks that are carried out at least weekly or monthly may appear 
to be important, while for others, tasks that require a higher level of skill 
complexity may be carried out rather infrequently and out of scope of the 
time span allowed for in the reference period. Answers to job-task questions 
are also likely to be affected by the response scales used. Response scales 
can be designed to question whether a job task is carried out at all (yes or 
no option), how frequently it is carried out (every day/at least once per week, 
etc.) or what share of a daily working time a task represents. The response 
scales may have to vary across different job task categories. Finally, the 
quality of responses to the job-task questions is linked to the interpretability 
of the formulation used in describing tasks and may be affected by the 
survey mode (face-to-face, computer-assisted telephone interviewing (CATI) 
or online) and environmental context (for example questions on carrying out 
specific ICT tasks may be affected by the recent coronavirus pandemic or 
other shocks). 


2.2.2. Capturing technological change 

As well as asking questions about skills, there is a need to think about 
how to capture information about technological change affecting workers’ 
jobs and skill needs. Previous efforts to do so have tried to measure the 
type of technological change being introduced, such as plant, machinery 
and equipment, or the introduction of new software or some combination 
of changes. In the absence of individual microdata, technological change 
is typically proxied with reference to investment in computer equipment 
(for example Autor et al., 2003), or firm automation costs defined as costs 
of third-party automation services (Bessen et al., 2019) or computer- 
related occupational job task changes (for example Deming and Noray, 
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2020). Such analyses typically combine data from several data sources 
to analyse change at the level of the job/occupation rather than the 
individual. 

In worker skill surveys, where the aim is to understand the impact of 
technological change, it is likely that its impact on jobs and skills will be 
shaped by a range of internal factors: 

(a) the type of technology being introduced; 

(b) the approach the company takes to introducing change (for example the 
extent of negotiation/consultation with workers); 

(c) previous experience of technological and organisational change; 

(d) the way in which the content of a job changes (job enlargement, job 
rotation, and job enrichment). 


It is likely that the respondent’s view of technological change will be 
affected by the overall context in which it is being introduced (for example, 
in an environment where there are job reductions versus one where 
there is dynamic job growth) and various extrinsic rewards attached to 
any charige (such as wages). In an ideal world, researchers would like 
to be able to compare the experiences of workers affected by the same 
types of technological change to gauge how much variation there is 
in their job-skill profiles. Being able to ask about specific technologies 
is likely to lie outside the scope of many respondents’ knowledge, but 
it may well be possible to ask questions on technological change with 
respect to: 

(a) its scale; 

(b) whether or not it is likely to replace routine job tasks; 

(c) the extent to which it is different from previous technologies; 

(d) how workers have been supported to acquire the skills needed to 
accommodate technological change (i.e. training and skill development). 


There may be general interest in asking workers about technological 
change and other changes affecting jobs and skills, as shown in Box 4. 
Or there may be an interest in approaching this more directly from 
a skills and learning/training perspective. These types of questions are 
used in the STAMP and Cedefop second ESJS surveys, as the example 
in Box 4 illustrates; they are particularly informative, as they can give an 
indication of the scale of the change that has taken place and its skill 
intensity. 


OO oO 


PREVIOUS CONTENTS NEXT 


CHAPTER 2. 
The art of asking questions: employer and employee skills surveys 


Box 4. Questions on technological change at work 


In the last five years or since you started your main job, have any of these changes 

taken place in your workplace? 

e Changes to the technologies you use (for example machinery, ICT systems) 

e Changes to your working methods and practices (for example how you are man- 
aged or how you work) 

e Changes to the products/services you help to produce 

e Changes to the amount of contact you have with clients or customers (for example 
dealing with customer/client queries or complaints) 


Source: Cedefop first European skills and jobs survey. 


In the past three years/since you started your current position/since you started this 

job) have you started using any new computer equipment or computerised machinery? 

e If yes: Now, think about the one piece of new equipment or machinery that took 
the longest to learn how to use. About how long would you say it took you to learn 
it: less than one week, one week to one month, between one and six months, six 
months to a year, or more than a year? 


Source: STAMP survey. 


[In the last 12 months IF tenure>=1 year] [Since you started your main job IF ten- 
ure<1 year], did you have to learn to use any new computer programmes or software 
to do your main job? By new we mean those you started using for your main job [in 
the last 12 months IF tenure>=1 year]. Please exclude minor or regular updates. 


Source: Cedefop second European skills and jobs survey. 


When putting questions to workers about the introduction of new 
technologies, it is important to clarify a number of issues to obtain sensible 
and robust answers. Careful definitions need to be provided regarding 
what constitutes technological change, for instance if the concept refers to 
new computer software, or ICT equipment or also includes computerised 
machinery and computerised equipment or something else. Other important 
survey design decisions include: 

(a) specifying whether a respondent had previously used or not the specific 
technologies at work, in other jobs or in other life situations (for example 
for recreation or social purposes), or if the interest lies only in new 
technologies that have recently been adopted in a workplace or used as 
part of a worker’s job; 
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(b) deciding whether the survey focus should include minor updates to 
existing technologies or if interest lies only in major episodes of change. 
Many estimates of technological change could be inflated as they capture 
minor updates to software or equipment that does not lead to marked 
changes in job-skill requirements; 

(c) considering whether to ask directly about whether the use of new 
technology was also accompanied by a need for learning, signifying that 
it entailed a skills-augmenting element; 

(d) specifying a clear and relatively recent time window (for example past 
12 months, past three years) in which a new technology that could have 
potentially affected skill requirements was introduced. 


Besides asking about technological and other changes in general terms, 
there may also be an interest in a specific technology, such as computers, 
robots and Al. Questions related to these need to be drafted in such a way 
that they make sense to the respondent. Not everyone, for instance, knows 
what an algorithm is, or what constitutes a robot or Al. The types of questions 
that can be asked are usually of the type presented in Box 5. 


Box 5. Question on specific technologies at work 


In the last three years, did any of the following changes occur at your workplace? 
e New computerised or automated equipment was introduced into the workplace. 
e New communications technology equipment was introduced into the workplace. 
e Robots were introduced into the workplace. 


To what extent are you familiar with the following technologies? 
(a) Virtual/augmented reality 

(b) Big data 

(c) Artificial intelligence 

(d) Internet of things 

(e) Internet of services 

(f) 3D printing 

(g) Blockchain technology 

(h) Quantum computing 

(i) Other technologies 


Source: Cedefop. 
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Information on skill levels can then be related to questions about 
technological change — and other types of change — in the workplace 
over a given period of time, and the respondent’s experience of those changes. 

In thinking about how to capture information on skills and technological 
change, the reader is referred to Cedefop’s European skills and jobs survey 
(Cedefop, 2015; 2018). This is an exemplar employee survey, designed for 
the European Union, aiming to collect detailed information on a wide range of 
skills needed in jobs, alongside other information about changes in jobs over 
time and workers’ experience of technological change (Box 6). 


Box 6. The Cedefop European skills and job surveys 


The first European skills and jobs survey adopted an innovative approach to col- 
lecting information on an individual worker’s skills, in that it asked them about their 
current skills and how these have changed, and why, since they started doing the 
job. This provides the opportunity to look at how a range of factors have influenced 
the development (or not) of an individual’s skills set. Detailed information on the job/ 
occupation of the worker was also collected, alongside that on specific skills (their 
importance to the job and workers’ assessment of how good they are at those skills). 


In summary, the first ESJS survey captured information about: 

e the current job (occupation in which the person is employed); 

¢ educational attainment (highest qualification held and field of study, and the qual- 
ification level required if entering the job today); 

e basic skills (workers’ ratings of their proficiency with respect to various skills sets, 
including numeracy, literacy and ICT skills); 

e change in skills since start of current job (whether skill levels have improved/ 
worsened, whether skills have remained matched to the job, whether the worker 
has engaged in activities to improve skills); 

e training (access to various types of training). 


The survey also asked whether people have been exposed to technological change, 
in particular if they had been subject to changes to the technologies used (for exam- 
ple machinery, ICT systems) or to their working methods and practices in the recent 
past. Such data can be analysed to reveal how the impact of technological change, 
other things being equal, affects the demand for skills and, importantly, which skills 
are affected and whether the impact is to increase or reduce the use of certain skills 
(McGuinness et al. 2019). In this way policy-makers can obtain detailed information 
on both the specific jobs and the specific skills which have been affected by tech- 
nological change (see Cedefop, 2015; 2018; Pouliakas, 2018). 
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To address the ongoing policy debate about the potentially enriching or destructive 
impact of digitalisation on jobs and the future of work, as well as heightened con- 
cerns about what may be a non-transitory, long-term impact of the Covid-19 crisis 
on EU job markets, Cedefop initiated the second ESJS wave in 2021. 


The second wave of the ESJS collects comparative information from EU Member 
States (plus Norway and Iceland) and some EU acceding countries on the impact of 
technological change and digitalisation on workers’ job tasks and skill mismatch and 
their readiness to adapt by investing in continuing (online) vocational education and 
training. Policy-relevant information on the incidence of teleworking in EU job markets, 
the degree to which employees have recently adopted new digital technologies, and 
the extent to which they have invested in non-formal and informal learning to cope also 
with the consequences of the coronavirus pandemic, is collected. Such information can 
be superimposed on the degree to which workers’ job-tasks are routine or involve high 
cognitive requirements and interpersonal skills, building on the dedicated task module 
that collects information on workers’ job-skill requirements. The survey also collects 
information on workers’ perceived job insecurity due to automation and skills-displac- 
ing technological change. 


Source: Cedefop: https://www.cedefop.europa.eu/en/events-and-projects/projects/european-skills-and-jobs- 
survey-esjs 


In uncovering the relationship between technological change and skills, 
it is important to be aware of the limitations of worker surveys. Workers can 
report if and how they have been affected by technology adopted within 
their workplace, including whether technology has caused them to lose their 
job recently, whether it leads them to anticipate possibly losing their jobs 
or relevance of their skills in the next year, or if it has changed the task 
composition of their jobs recently. However, changes in the composition of 
employment can be driven by shifts in employment across establishments. 
If the workforce in more technologically backward establishments contracts, 
due to competition from more advanced establishments, it is likely that the 
displaced workers will be unaware of the underlying reason for their job loss. 

It may also be the case that there are potential spillover effects (at industry 
or workplace level) due to new technologies for the individual workers, even 
though, in their view, technological change has not had direct impact on the 
part of the company in which they are based. It is reasonable for a survey 
to ask workers about their own workplace, but they cannot generally be 
expected to have detailed knowledge of other workplaces or system-level 
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dynamics. To detect these kinds of technology-induced shifts would require 
data on ICT adoption or investment by industry at a sufficiently detailed level. 

Table 4 provides a summary of the issues to consider when thinking 
about using a survey of employees/individuals to explore the link between 
technological change and skill levels. It provides a checklist of what needs 
to be considered before commissioning such a survey. 


Table 4. Issues to consider before launching an individual/worker skills 
and jobs survey 


‘issue | Detail to be considered 


Initial check Do similar data already exist elsewhere? 

Types of e Have individuals experienced technological change in their jobs (e.g. over 
questions the past 12 months has the introduction of new technology significantly 
that can be changed the way you do your job?)? 


addressed about 
technologies 


e Have individuals recently used specific technologies (e.g. computers, 
robots, Al) in their jobs? 


Types of 
questions 

that can be 
addressed about 
skills 


e Rating of individuals’ own skills compared with those required to 
undertake the job — this can be disaggregated to ask about specific skills 
(e.g. numeracy, literacy, ICT) 

Level of specific skills 

Frequency of specific skills use 

Importance of specific skills for doing the job 

Complexity of skills 

e Change in use of specific skills 


Requirements Sampling frame from which to derive a representative sample of employees/ 
individuals in which there is an interest 
Data collection § ° Specifying hypotheses to be addressed 
steps e Specifying the variables and their relationships required to test the 
hypotheses 
e Drafting questions that will allow variables to be constructed (the questionnaire) 
e Crosschecking questions with other surveys (to allow for comparisons) 
e Design of sample (random probability, quota sampling, etc.) and sample 
size required for robust estimates to be derived 
e Type of survey (face-to-face, telephone, online, etc.) 
e Fieldwork 
e Weighting dataset to population estimates 
e Data cleaning 
Selected further © European skills and jobs survey 
literature e UK skills and employment survey 


Source: Cedefop. 


e Handel, M. (2017). Measuring job content: skills, technology, and 
management practices. In: Buchanan, J. et al. (eds). The Oxford handbook 
of skills and training. Oxford: Oxford University Press, pp. 92-123. 
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2.3. Skills Surveys: a Summary 


The type of information skills surveys can provide (Table 5) is not focused 
on the future per se, but can be of value in revealing the direction of change. 
The key point is that surveys among employers and employees can provide 
detailed information on how skill demands are changing in the workplace 
and on the drivers of change. These surveys demonstrate the way in which 
technological change’s impact on skills is mediated to some extent through 
work organisation, where employers have a degree of strategic choice. It is 
not necessarily the case that technology x will always create a demand for 
skill y: ultimately the impact depends on the way in which employers decide 
to configure jobs (occupations) with respect to the combination of tasks they 
expect the employee to undertake. 


Table 5. Typical information skills surveys can provide about 
technological change and skills needs 


Employee skill surveys Employer skill surveys 


Information about skill content of jobs Extent of technological change 
taking place in workplaces 
Changes in the skill content of jobs Impact of technological change 


on skills proficiency of existing employees 
Impact of technological change, other things Extent to which technological change 


being equal, on skill content of jobs affects recruitment from external labour market 
(skill shortages) 

Information on workplace factors that have Whether technological change is supported 

some bearing on the impact of technological by reskilling and training of employees 


change on skill content of jobs 


Understanding the role of training in mitigating 
the impact of technological change on skills 


Source: Cedefop. 


There are difficulties in asking questions about the future in questionnaire 
surveys; this is especially so for surveys among workers who may not be 
well placed to provide an informed view about future developments, other 
than those their employer has told them about. In employer surveys there is 
evidence suggesting future technological changes may be underestimated. 
Surveys are generally much better at delivering information about recent 
changes and the current situation. 
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Healthy scepticism of studies that claim to discern the future by asking 
respondents what changes they anticipate in their jobs or firms in the next 
five or 10 years is appropriate. A good principle of survey research is to 
avoid asking people questions they are unable to answer, or at least answer 
reasonably easily. Some of the risks in asking people to imagine future 
conditions can be illustrated by the World Economic Forum’s (WEF) Future 
of jobs survey, which surveyed chief human resources and chief executive 
officers in leading global employers from November 2017 to July 2018, 
regarding their current planning and projections related to jobs and skills in 
the period leading up to 2022 (World Economic Forum, 2018). Among the 
results for the adoption of ICTs: 

(a) 36% of executives thought it likely or very likely their company would 
adopt quantum computing in the next five years; 

(b) 23% anticipated adopting humanoid robots; 

(c) 41% thought it likely they would adopt 3D printing; 

(d) 45% anticipated adopting blockchain technology (World Economic 

Forum, 2018). 


Although the anticipated adoption rates for more mundane ICTs were 
more plausible (for example 72% expecting to adopt cloud computing), it is 
clear that particular types of respondent given room to use their imagination 
regarding the future, will do exactly that. 

A less extreme example of this problem occurred when the US Census 
Bureau conducted the Survey of manufacturing technology (SMT). In 1988, 
the SMT asked plant managers about the presence of 10 automation 
technologies and their plans for implementing them in the following five 
years if they did not use them currently. When the next (and final) wave 
of the survey was conducted in 1993, actual prevalence rates for most of 
these technologies were 10 to 12 percentage points lower than the rates 
implied by the responses to the intend-to-implement questions asked five 
years earlier (Handel, 2004). Clearly, both the WEF and SMT surveys exhibit 
a significant bias toward overstating the prospective rate of technological 
change over a five-year period, suggesting moderate to serious problems 
with validity. 

In contrast to these requests for medium-term speculations from 
respondents, it is useful in terms of technology forecasting and skills 
anticipation to ask workers whether they have concrete knowledge of 
technological or other changes in their workplace or the broader labour 
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market in the next 12 months, that are likely to affect job security, required 
skills, or other relevant outcomes in foreseeable ways. 

In relation to technological skills obsolescence and its impact on 
employment, longitudinal data are needed to demonstrate how technological 
change affects skill demand in a particular workplace or the chances of 
a worker remaining in employment. To date, these types of data are scarce, 
mainly because of the costs involved in carrying out panel studies and 
the problems attached to sample attrition over a relatively long period of 
time. The construction of pseudo-cohorts within a survey can alleviate this 
to some extent, but the results will be dependent upon the recollection of 
respondents. For example, it is possible to construct a survey of people of 
different ages who are economically active and ask them about their recent 
employment history (career history). 

When deciding to carry out skills surveys, key questions to address are 
why the survey is needed and if there is an existing survey that already 
provides the data. This may seem obvious, but there has been a proliferation 
of surveys over recent years such that would-be respondents may face 
survey fatigue. This can be especially true for employer surveys where those 
workplaces which meet certain criteria — typically large ones which employ 
many people - are relatively few and therefore tend to receive multiple 
requests for information. 

Skills surveys should also ensure that the respondent is able to answer 
the questions on technological change and skills easily. Speculative and 
prospective questions, or those on generic types of technologies (for 
example artificial intelligence) that some individuals or human resource 
managers (especially in small and medium-sized businesses) are unaware or 
imperfectly aware of, may give rise to unreliable data collection. 
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3.1. Modelling technological change 


Skills forecasting involves the use of relatively elaborate economic models 
that provide insights into future developments in the structure of employment, 
skills supply and demand. This is usually derived from economic relationships 
in time series analysis. Said simply, forecasting can be thought of as 
extending trend lines. 

Figure 1 provides a stylised extrapolation of the future based on the past. 
Past observations, depicted as dots, can be analysed using approaches such 
as regression techniques, which allow the establishment of relationships, in 
this case, over time. The outcome variable could, for example, be employment 
levels (in an economy, sector, or occupation). If a relationship is consistent 
over time, it can be extrapolated into the future, depicted by the dotted line. 
This will work as long as the identified relationships continue into the future 
without any structural breaks or shocks (°). 

If the possibility of technological change is introduced, which is not part 
of the model, the underlying relationship based on which the original forecast 
is made might no longer hold. Figure 2 shows how the prediction after 
technological change may lead to higher (or in some cases lower) general 
levels of the outcome variable of interest. Technological change might even 
speed up the employment effect (affecting the slope of the line), while in 
other cases the impact might be temporary or wear off after a period of time 
(Figure 3). 


(®) This simplistic version should be seen as representative of many estimated relationships 
between one or more variables that can be either consistent or evolving over time. An important 
requirement is that there is no structural break or shock to the relationship (or to the underlying 
data) for the forecast to continue to hold with some degree of accuracy. 
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Figure 1. Stylised extrapolation of future trend 
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Source: Cedefop elaboration. 


Figure 2. Stylised extrapolation of future trend with technological shift 
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Source: Cedefop elaboration. 
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Figure 3. Stylised extrapolation of future trend with dynamic changes 
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Source: Cedefop elaboration. 


Any analysis of technological change in the context of skills forecasting 
poses the following challenges: 
(a) need to identify that there is a new technology that affects the development 
of the variable of interest over time (the ‘relationship’); 
(b) uncover the way technology affects the estimated relationships (is it 
a shift, as in Figure 2, or is it an increase of dynamics as in Figure 37); 
(c) describe the process whereby technology affects estimated/underlying 
relationships in the model (does the effect continue over time? Does it 
slow down or diminish’). 


Even with an identifiable technology and understanding of the way it 
affects specific relationships, estimating the influence of the technology 
in the future can still be challenging. The main problem for the forecasting 
process is a need for observations over time; in many cases there are 
no historical data and the influence and importance of technology only 
becomes apparent ex-post, which may be too late from a skills anticipation 
perspective. 
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3.2. Modelling skill demand 


Quantitative modelling approaches to anticipating future skill needs include: 

(a) univariate models, such as simple extrapolation of past trends (mechanistic 
techniques) and more complex time series methods; 

(b) complex multivariate time-series-based behavioural/econometric models. 


Extrapolation techniques are often used when only very limited time 
series information is available, which clearly restricts the sophistication of 
the analyses. Where longer time series of observations are available, more 
complex analysis may be possible; here, replicable patterns in a time series 
may be found and used to predict its future path. Such approaches are widely 
used in the business and financial world, though they are much better at 
predicting short-term change than longer-term patterns. Unfortunately, most 
linear (or more complex) trend patterns eventually come to an end (trends 
bend) in their ability to predict the true developments over a longer time. 
Therefore, they should not be relied on for medium- to long-term forecasting. 

The typical quantitative modelling approach involves two key elements. 
The first key component is a multisectoral macroeconomic model of some 
kind — usually built around a Leontief input-output table — that takes into 
account the interlinkages between sectors. Such models are usually 
estimated using complex econometric methods, although computable 
general equilibrium (CGE) models (where parameters are imposed rather 
than estimated) are also used. Most skills forecasting is carried out using 
econometric models. 

Econometric models are subsequently built using sophisticated statistical 
and econometric techniques, and using large datasets drawn from official 
sources, including national accounts and estimates of employment based on 
employer and household surveys. A typical multisectoral macroeconometric 
model comprises many equations. Parameters are estimated using multivariate 
econometric techniques, based on error-correction and related time-series 
methods or Heckman correction methods on cross-sectional data. 

The model typically has two main components: 

(a) a demand for skills module; 
(b) a supply of skills module. 


Figure 4 presents the modelling framework, showing where the supply 
and demand components meet. 
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Figure 4. Modelling framework underlying skill forecasts 


DEMAND-SIDE 


SUPPLY-SIDE 


Source: Cedefop elaboration. 


The key steps in skills forecasting are common and typically include: 

(a) a projection of employment prospects in the main sectors or industries; 

(b) translation of these projections into trends in employment at the level of 
occupations; 

(c) an assessment of replacement needs (to take account of job openings 
arising from retirements and other withdrawals from the workforce); 

(d) forecasts of supply of labour by age, gender, qualification level and 
occupation, encompassing both new entrants to the workforce and the 
unemployed; 

(e) calculation of (future) labour market imbalances by comparing occupational 
demand with various indicators of supply. 
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3.3. Advantages of skill forecasts over 
alternatives 


The main advantage of the macroeconomic modelling approach to skills 

forecasting is its simplicity (if a macroeconomic model already exists). 

It is essentially a method enabling the disaggregation of an existing 

macroeconomic employment forecast and identification of the implications 

for skills demand in a relatively quick and easy manner. Similarly, on the 

supply side, demographic forecasts can be used to predict what the future 

holds for working age population numbers and their respective qualifications. 
In general, quantitative skills forecasting models should be used to make 

projections because they: 

(a) make assumptions about the future explicit and transparent; 

(b) enforce systematic and logical thinking; 

(c) act as a focus for intelligent debate (°); 

(d) provide a useful counterfactual (to anyone interested in possible future 
outcomes) against which to assess policy impacts (i.e. what would have 
happened in the absence of a policy intervention). 


They should not be relied on for detailed employment planning or to 
give precise indications of education and training requirements, as the 
extrapolations are necessarily simplified and cannot identify exact numbers. 

Most reviews of international best practice in skills forecasting suggest 
that the use of a national multisectoral macroeconomic model-based 
approach is the preferred option (’). Such models are regarded as essential 
in order to obtain a robust and consistent sectoral employment scenario, 
which is the starting point for any comprehensive assessment of changing 
technologies and skill needs. 

The advantages of such a multisectoral approach are: 

a) the sectoral and other detail such a model provides; 

b) the fact that it is typically comprehensive, covering the whole economy; 
c) logical consistency; 

d) imposition of accounting constraints; 


( 
( 
( 
( 


(°) Forecasts should not be seen as an end in themselves but rather as a focus for informed debate. 
They are a considered view of what might happen next. Without some kind of firm statistical 
foundation, all such debate is mere speculation. 

() See ETF et al., 2016a; 2016b and 2016c. 
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(e) recognition of economic constraints and influences; 
(f) the fact that it helps make underlying assumptions explicit; 
(g) provision of consistent scenarios across all sectors. 


Different macroeconomic forecasts for employment can, in principle, 
also be compared for their implications for skills demand, as can different 
population projections. In addition, changes can be made to model 
assumptions, such as the proportion of different occupations across sectors, 
or qualifications in occupations, to see what difference this would make to 
the overall findings. 

As with any skills anticipation method, macroeconomic forecasts also 
have disadvantages, limitations and problems. These relate to: 

(a) technical limitations, within fixed resource limits (°); 

(b) limits to current understanding of the way labour markets work; 

(c) the possibly limited relevance of the past (Such models being based 
on assuming continuation of past patterns of behaviour). This may be 

a pronounced issue in the context of countries subject to significant 

political and social disruption; 

(d) resource costs of development and maintenance (°). 


The data requirements of quantitative modelling approaches are substantial. 
They include a sufficiently long time series of consistent data on a range of 
economic and labour market indicators (especially on employment, skills, 
and labour supply), as well as input-output tables. Sectoral employment 
data lie at the heart of any multisectoral modelling approach to assessing 
changing skill needs. Ideally such data need to be linked to other economic 
indicators within a system of national accounts ('°). Developing the necessary 
data infrastructure requires many years of substantial investment. 

There are also limitations arising from the simple fact that often the 
data used to build models were not collected with modelling in mind: 


(®) With unlimited budgets, much better data can be collected, and better and more complex 
models can be built, but there are limits to our understanding, so there is no guarantee these 
models will be able to predict the future any better than simpler models. 

(°°) These costs include those directly concerned with building and executing a skill forecasting 

project. This is distinct from the statistical infrastructure necessary to support such activity, 

which requires a much more long-term commitment. The former is modest compared to the 
latter but can still be substantial. 

A fully specified multisectoral macro model requires a large amount of economic and labour 

market data on each sector. 
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most economic and labour market data support tax collection and other 
government/administrative objectives. They are rarely, if ever, collected with 
the objective of being able to build economic or other models to be used 
specifically for skills assessment and forecasting. 

A forecast model needs to be based on a number of macroeconomic 
assumptions to make it operational. This includes explicit assumptions, such 
as population forecasts by cohort, as well as more subtle assumptions, such 
as the use of shares of the informal economy. 

With simplicity, however, comes limits. Because the modelling system is 
effectively a top-down approach, it cannot deal with more complex, sector 
or policy-specific questions that require a more bottom-up methodology. It is 
important to start with a relatively simple framework, so that understanding 
and knowledge can be established as a base upon which to build more 
complex representations of the economy. The main contribution of using 
a model-based framework to reflect on the demand for and supply of labour 
is that it enables establishing a baseline of skills profiles and trends. It is 
crucial to keep in mind that any image of the future, as sketched by models, 
is limited by the nature of their construction. 

Quantitative models should not, therefore, be seen as a panacea for 
understanding the future of jobs and skills. Nevertheless, in most of the 
countries that conduct regular national assessments of future occupational 
and _ skill requirements (''), such models are regarded as an essential 
cornerstone. Quantitative modelling is increasingly being adopted in 
developing, as well as developed, countries, as data availability and the 
capacity for model building improves ('*). 


3.4. Incorporating technology in a skill forecasting 
modelling framework 


As the estimated relationships within the macroeconomic model rely mainly 
on historical data, and the assumption is made that they will apply in the 
future, this also implies that only technological changes or technologies 
already present can be expected to be identified and included in the 
modelling relationships. 


(") This includes most members of the OECD and the EU. 
('2) For a review see, for example, ETF et al., 2016a; 2016b and 2016c. 
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Including technology or technological change that affects the labour 
market, and through it the skill mismatch outcome, faces certain problems. 
As identified, traditional skills forecasting methods presume past experience 
will extend to future developments. To include technology which is not 
yet incorporated into the modelling framework, an analyst would need to 
consider key steps. 

First, the technology that can impact future needs has to be identified. 
This is necessary especially if the technology has an impact that cannot be 
anticipated from its past development. Technologies that evolve over time, 
for which past development is a good measure of future developments, 
may be captured well within quantitative models. The problem arises if the 
technology is new, if it is evolving at a speed not anticipated, or is impacting 
parts of the economy previously unaffected by it. An assessment must be 
made about whether the new technology comprises a structural break. 
Important questions to be asked include: what occupations or sectors are 
most likely to be affected by such new technological developments? Will it 
spill over into new, yet unaffected areas within the anticipation period? 

Second, the areas in which the technology affects the model need 
to be identified. This can be done within the macroeconomic model, 
affecting both supply and demand components, overall economic 
growth, sectoral employment changes, and also the consumption side. It 
can affect the occupational structure within sectors, as technology might 
change the production process of goods and services in such a way that, 
for instance, more automatisation encourages a shift from intermediate 
level production workers towards higher-level engineering. However, 
there can also be lateral substitution through technology from one type of 
occupation to another. 

Third, it is necessary to gain insight into the dynamism of the technological 
development. In short, what is the speed of development and its impact on 
the labour market and skills for the period of anticipation? The speed of 
these changes usually cannot be identified from the past, as they depend on 
patterns of technology adoption and the diffusion of work practices that are 
not easily known or estimated. 

Insummary, including technological changes into a skill forecasting model is 
challenging. Dedicated data sources that can identify technological change are 
typically lacking. Such gaps are usually overcome by relying on a combination 
of qualitative and quantitative evidence, coupled with assumptions used to 
estimate impacts on employment and skills demand. A much-debated early 
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attempt to identify changes in occupational demand due to technological 
change by Frey and Osborne (2017) is discussed in Box 7 ('%). 


Box 7. Extrapolating future skill demand based on external evaluations 
of technological change 


The well-cited study of Frey and Osborne (2013; 2017) provides an estimate of the 
degree to which occupations may be replaced by automation. The basis for this 
prediction is expert evaluations of the possibility of replacement of machine-learn- 
ing algorithms for labour in a specific subset of all occupations. These evaluations 
(a training data set) were then extrapolated using specific aspects of occupations 
linked to their task features and suitable non-parametric regression techniques. 
Based on the extrapolation, an automation probability was derived. While the com- 
bination of expert knowledge on the degree to which specific occupations might be 
automated seems sensible, the approach remains one-sided: it only provides an 
estimate of how much a task or job can be automated. It does not, however, take into 
account the economic feasibility of this actually being done — which can be a cost 
and employer incentives question (Cedefop, 2020) — as well as the equilibrium ef- 
fects of such automation. It is likely that additional work will be generated following 
machine replacement, given that the overall output is produced more cheaply, or 
that different tasks are required; both could lead to additional employment effects. 
Other studies on the effects of automation on the occupational structure have tried to 
take some of these secondary effects into account. Using data on tasks and skill needs 
in jobs, collected in Cedefop’s European skills and jobs survey, Pouliakas (2018) esti- 
mates the empirical relationship between the variance in skill needs within occupations 
and the probability of automation, on the basis of which jobs are classified according to 
their estimated risk of automation. A job is considered as subject to a very high risk of 
automation when the median automation probability estimated exceeds 70%. The num- 
ber of jobs by occupation at high risk of automation in the future can then be estimated 
by superimposing the share of people working in the high-risk occupations onto the 
occupational projections derived from an external skill forecasting exercise (Figure 5). 
These approaches are partial and non-endogenous: without considering the supply-de- 
mand interactions in the macroeconomic model, effects can be difficult to estimate. This 
becomes even more of a challenge as skills forecasting in many countries relies on in- 
creasingly complex macroeconomic models, making updating of relevant estimations to 
model technological changes properly a complex and demanding exercise. 


Source: Cedefop. 


('3) While the study was published in 2017, initial versions circulated in 2013. 
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Figure 5. Occupations at risk of automation in the period to 2030 


Code 
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Chief executives, senior officials and legislators 
Administrative and commercial managers 
Production and specialised services managers 
Hospitality, retail and other services managers 
Professionals 

Science and engineering professionals 

Health professionals 

Teaching professionals 


Business and administration professional 


Information and communications technology professionals 


Legal, social and cultural professionals 
Technicians and associate professionals 
Science and engineering associate professionals 
Health associate professionals 

Business and administration associate professionals 
Legal, social, cultural and related associates 
Information and communications technicians 
Clerical support workers 

General and keyboard clerks 

Customer services clerks 

Numerical and material recording clerks 

Other clerical support workers 

Service and market sales workers 
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Code Occupation Share of people at very high risk of automation 
5.52 Sales workers 
5.53 Personal care workers 
5.54 Protective services workers 
6 Skilled agricultural, forestry and fishing workers 
6.61 Market-oriented skilled agricultural workers 
6.62 Market-oriented skilled forestry, fishery 
6.63 Subsistence farmers, fishers, hunters and gatherers 
7 Building, crafts or related trades workers 
771 Building and related trades workers 
7.72 Metal, machinery and related trades work 
7.73 Handicraft and printing workers 
7.74 Electrical and electronic trades workers 
7.75 Food processing, wood, garment and related workers 
8 Plant and machine operators and assemblers 
8.81 Stationary plant and machine operators 
8.82 Assemblers 
8.83 Drivers and mobile plant operators 
9 Elementary 
9.91 Cleaners and helpers 
9.92 Agricultural, forestry and fishery labourers 
9.93 Labourers in mining, construction, manufacturing 
9.94 Food preparation assistants 
9.95 Street and related sales and service workers 
9.96 Refuse workers and other elementary workers 


NB: The red line represents the average value of automation risk in the EU. 
Source: Estimates of Pouliakas (2018) superimposed on Cedefop skill forecasts. 
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In essence, technological change can be included in macroeconomic 
models by either explicitly modelling the change process or by adjusting 
assumptions (often parameters) and relationships in the model, often 
based on qualitative insights (Box 8). Technological change, especially in 
its productivity-enhancing element, is regularly included in models. The 
resulting higher productivity can also be observed from monitoring statistical 
data over time. By developing the relationships that pick up and model the 
impact of technological change, more insights can be gained by varying 
the implied relationship (i.e. sensitivity analysis). For example, if there is an 
expected shift in the productivity-enhancement effect of a new technology, 
the increase in productivity could be captured by the model by integrating 
such an expectation. 


Box 8. Combining qualitative and quantitative inputs in the German 
forecast 


In the German forecast up to 2030, a digitalisation scenario was developed, based 
on qualitative in-depth studies on specific aspects of digitalisation and automation. 
Besides the baseline scenario, an accelerated digitalisation scenario was developed 
to assess the impact of increasing digitalisation on the labour market. The qualitative 
studies provided necessary inputs for altering assumptions within the macroeco- 
nomic model at the sectoral level, leading to changes in demand for occupation and 
skills within sectors. 

In many instances, lack of historical data on the effects of changes had to be over- 
come by including information from the qualitative studies. The qualitative informa- 
tion had to be translated into quantitative estimates that could feed into the mac- 
roeconomic model, i.e. sector-based employment effects, and into changes of the 
occupation and/or qualification effects of digitalisation. Some of these were ad hoc 
and expert-based, while others could be grounded more in the information the qual- 
itative studies provided. 


Source: Cedefop. 


Occupation and qualification forecasts, which build on sectoral forecasts, 
also rely on identifying historic developments that are expected to continue 
in the future. Digitalisation and technological change potentially affect work 
organisation within sectors. As long as these changes can be identified and 
occur gradually as time passes, they can be included in the occupation 
and qualification modules of the skills anticipation model. Trends that 
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occur rapidly, that have no precedence or follow a pattern that is difficult to 
predict, are much more difficult to incorporate. As in technology adoption 
models, attempts can be made to model the speed of change. However, the 
identification of these relationships is usually well beyond the scope of what 
can be achieved in modelling, and data are usually insufficiently detailed and 
reliable to fit ‘technology-adoption’ types of models. 

Another problem is the emergence of new occupations or skills that are 
neither classified in the statistical data nor existed in the past. The web- 
developer occupation did not exist before the 1990s, and the growing 
employment in this field and trends in associated skills and qualifications were 
initially picked up only in related occupations, given that these positions were 
filled by people with qualifications suitable for the tasks to be performed. 

Given that technological changes are difficult to estimate and identify 
using past data, in many cases model builders rely on using multiple, partial 
sources to adjust assumptions within the model. To identify the impact of 
these changes, running and forecasting different scenarios is currently the 
best available approach to identifying labour market changes implied by 
technology and digitalisation. 


3.5. Limits to skill forecasting with technological 
change 


While skills forecasting can provide useful insights by combining historic 
trends to shed light on current and likely future developments of skills supply 
and demand, it falls short when new developments occur that cannot easily 
be identified and included in models representing economic relationships. 
The identification of emerging or future technological change is crucial to its 
inclusion in such models. But, as the methods have their foundation in historic 
developments, they tend to miss predicting outcomes that are entirely novel, 
skills that have not been used in the past, or occupations that are yet to 
be defined. Methods that complement skills forecasts models, such as big 
data, foresight and other types of skills anticipation, are increasingly used to 
shed some light on likely future trends driven by technological change. 
Table 6 provides a summary of the issues to consider when thinking 
about engaging in the econometric forecasting of skill needs in relation to 
technological change. It provides a checklist of what an analyst will need to 
consider before commissioning or engaging in a modelling exercise. 
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Table 6. Issues in developing a skill forecasting approach to the impact 
of technological change 


‘issue Detail to be considered 


Initial check e Can existing skill forecasts be extended to include technological change? 
e Is the technological change sufficiently identified and measurable using 
data from the past? 
Types of e What is the impact of technological change on labour demand and supply? 
questions e How do different speeds of technology adoption in the workplace (as seen 
that can be in scenarios) affect the economy differently? 


addressed about 
technologies 


Types of 
questions 

that can be 
addressed about 
skills 


e What is the impact of various external factors, such as technological 
change, on the demand for skill (typically using occupation and highest 
qualification as proxy measures of skill)? 

e What are the estimated replacement demands as well as the overall change 
in the number of people employed in an occupation? 


Requirements 


e Macroeconomic model with the possibility to include/model technological 
change at the macroeconomic and sectoral level 

e Detailed microeconomic data on employment by sector/occupation/skills 

e Detailed data series on technological change 


Modelling steps 


e Run the model without specific modelling of technological change 

¢ Determine technological change scenarios 

¢ Determine within the macroeconomic model how (different assumptions 
on) technological change determines model parameters; if necessary, re- 
estimate relationships that might be affected by technological change 

e Determine within the occupation/qualification modules how technological 
change affects the assumptions, speeds of adjustment and job 
requirements 

e lf necessary, adjust assumptions/modelling of replacement needs 

e Ensure realistic assumptions including supply/demand relationships and 
interactions 


Selected further 
literature 


Source: Cedefop. 


e ETF etal., 2016b: 

Developing skills foresights, scenarios and forecasts 
e Cedefop: 

Skills forecasts 
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In this first Cedefop guide on methods for identifying technological change 
and its impact on skill requirements, consideration has been given to what 
may be referred to as conventional labour market intelligence approaches. 
This includes employer and employee surveys to collect information about 
changing skill needs, and using skills forecasting to provide an indication of 
projected future changes. 

Major investments in skills anticipation over the past 20 years have 
prepared the groundwork for anticipating future skill demand. Significant 
advances have been made in capturing detailed information about how 
people’s jobs are changing in the face of numerous shifts in what goes on in 
the workplace. Surveys, such as Cedefop’s European skills and jobs survey 
and the OECD’s survey of adult skills, for instance, now provide detailed 
information on the skills required in jobs and how these skills have developed 
or changed in the recent past. Improvements in the econometric forecasting 
of future skill needs have helped provide a more detailed view of the future 
than was available at EU and Member-State levels at the time when skills 
forecasting became widely used. 

More recently, the use of big data and Al-driven techniques has supported 
developing granular data on skills and technologies that would have been 
unimaginable 20 or 30 years ago (see Cedefop’s second practical guide 
referred to in Section 1.5). These techniques have been used to harvest data 
from, among other sources, vacancy websites, patent and scientific paper 
databases, to give a previously unprecedented amount of detail on emerging 
technologies and skill needs. 

Databases of online job postings or other similar web-sourced data have 
especially become a rich source of information on job skill requirements 
and they are anticipated to become increasingly important in research 
going forward. But they will not eliminate the need for surveys designed to 
address specific research questions. The reasons are that the underlying 
data extracted from web sources are unstructured and not generated for 
research uses. As a result, any repurposing, or data classification and 
analysis carries uncertainties and limitations. Another major problem is the 
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issue of representativeness. For instance, online job portals do not cover 
many vacancies filled through word of mouth. Representativeness varies by 
occupation and coverage of different labour markets tends to be linked to 
data source type (for example high-skilled jobs in private web portals and 
blue-collar jobs in public employment service portals). Based on flow data, it 
is not clear whether vacancy postings are representative of the current stock 
of employment with respect to skill requirements; jobs with above-average 
turnover will be overrepresented relative to their share of employment. Single 
posts can represent multiple vacancies, or even no vacancy, given the low 
cost of online job listing and employers may post jobs online simply to see 
which potential candidates are available on the labour market (So-called 
‘ghost’ vacancies, see Cedefop, 2019). 

There are also challenges with the skills information itself collected 
in web-sourced data. A survey will use a common set of questions to all 
respondents in the universe and score responses on a common scale, 
while online job ads will tend to focus primarily on occupation-specific skills 
rather than transversal skill concepts. These skills can be quite specific and 
difficult to aggregate into a broadly applicable, common scale because 
they are qualitatively diverse and usually not easily mapped into a level of 
complexity framework, as is typically done in surveys employing a job-skill 
requirements approach. It is possible to code the presence or absence of 
a specific skill requirement (for example commercial truck driver’s license, 
strong problem-solving skills, biochemistry, work with robots), or count 
the number of skills of a given class (for example ICT-related skills) as 
they appear in job ads (for example the number of computer programmes 
required). Yet job advertisements typically may not specify all important skills 
and technologies explicitly; many are widespread and implicitly expected. 
Most online job advertisement databases also have little data on the 
characteristics of workers actually hired to fill jobs, which may differ from 
employers’ stated preferences in job ads, for instance in terms of education 
credentials, experience, and specific skills. 

Also, the algorithms for scraping and processing online postings and 
similar web-sourced data sources evolve, so trend studies will need to 
distinguish real change from statistical artefacts. By contrast, surveys can 
be repeated following standard procedures. 

While the value of conventional labour market and skills intelligence 
approaches will remain and should be used as first resort when the aim is to 
obtain methodologically sound empirical technology and skills relationships, 
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it is nevertheless clear that the volume of data on future skill needs extracted 
from other sources has improved immeasurably and will continue to do so. 
In many respects, the challenge is to make effective use of the wide variety 
of data available (an embarrassment of riches). Technology skills foresight 
(discussed in guide 3 of the series) also has a key role to play in making 
sense of information. Its key strength is analysing and critically assessing 
various scenarios of the future to inform policy. Implicit within the foresight 
process is its focus on policy-making. The strength of the foresight approach 
is a weakness at the same time. In some cases, a normative element that 
is too concerned with actively shaping the future rather than passively 
accepting what it will be is overly dominant. 

Participatory and quantitative, non-participatory methods are not 
mutually exclusive. Ideally, they should support one another so that they can 
potentially form an iterative process, whereby the participatory process of 
stakeholder engagement can shape data collection and analyses in the non- 
participatory ones (and vice versa). Such interaction makes it possible to 
develop views on how the future will unfold and how informed skills policies 
and actions need to develop. It provides those with a responsibility for skills 
anticipation in a technological turbulent world with the means to ensure 
that skills supply does not only meet demand but does so in a way that is 
considered optimal and in line with policy priorities. 

Table 7 provides a summary to guide policy-makers and analysis in 
understanding when to use the approaches covered by the short Cedefop 
‘how-to’ guides (see Section 1.5). To learn more about big data and foresight 
approaches, readers are referred to the other two guides. 


© 


PREVIOUS 


®) 


CONTENTS NEXT 


©) 


CHAPTER 4. 
Choice of method | 53 


Table 7. A menu of skills assessment and anticipation choices 


Type of Capacity to Timeliness 
approach predict the future 


Quantitative, non-participatory approaches 


Surveys When there is a relatively | Tend to be good at Can be time- 
and other well-developed collecting information consuming to 
primary data understanding of the about recent past and undertake — design 
collections technologies and impending changes. Not of questionnaires, 
associated skills of well suited to anticipating | conducting fieldwork, 
interest. Surveys will tend | future technological Cleaning data, 
to provide information on changes and future skill producing findings. 
the extent of use of skills | needs. 
and technologies, extent to 
which skills are available, 
efforts taken to fulfil skill 
needs, etc. 
Skills Where time series data are | Skills forecasting models If the model already 
forecasting available on skill needs tend to provide a projection | exists, analysis can 
(based on qualification and | of future demand, based be undertaken over 
occupation), and where on an extrapolation of a relatively short 
there is an underlying past trends and/or current | space of time. But 
macroeconomic model policy. The assumption is _| setting up the initial 
that can provide robust that the future is based on | model and ensuring 
estimates of future a continuation of current regular updates of 
employment demand by trends. Scenarios provide | the results can be 
sector, skills forecasts can | some basis for varying this | time-consuming and 
provide a robust means to some extent, to account | resource-intensive. 
of providing quantitative for continual technological 
projections of future skill change. 
demand (circa 10 years 
ahead) 
Big data Particularly useful where Can provide relatively Can be time- 
analysis views about the future may | real-time information on consuming to 


not be well developed: 
where there is uncertainty 
about either the types of 
technology that are likely 
to become dominant or 
commonplace, and/or the 
skills associated with those 
technologies. Can also 
provide the detailed level 
of analysis that forecasting 
and surveys struggle to 
provide. 


technological change and 
skill needs. By identifying 
those technologies that are 
at the point of take-off, there 
is scope to gauge likely 
future skill needs. There are 
uncertainties about how 
representative data are of 

a given population and about 
how much ‘noise’ can be 
removed from any analysis 
or their inability to provide 
standardised information on 
skills complexity. 


develop initial search 
algorithms, but 

once established 

can be undertaken 

in a relatively fast 
manner. It needs to 
be borne in mind that 
coding/classifying 

of technology and 
skills data can be 
time-consuming. 
Maintenance and 
operational costs are 
also non-trivial. 
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Type of Capacity to Timeliness 
approach predict the future 


Participatory approaches 


Technology 
foresight 


Source: Cedefop. 


Where there is a large 
amount of information 

that needs synthesising 

to develop actions to 
ensure that skills needs, 
associated with particular 
technologies, can be met. 
Where there is limited data 
and information and where 
expert groups can address 
the lack of information. 


Can provide a view of the 
future and, importantly, 

an indication of how the 
future might be shaped for 
the benefit of society as 

a whole. Is dependent upon 
the availability of expert 
groups who can provide 
key input and a process in 
place to develop a degree 
of consensus about the 
future direction of change. 


Depends upon 

the scale of the 
exercise. Full-scale 
foresight involving 

a large number of 
participants is likely 
to prove time- 
consuming. But it is 
possible, and at times 
advisable, to conduct 
foresight with 

smaller groups over 
a relatively short-time 
span. 
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Acronyms 


Al artificial intelligence 

CATI computer-assisted telephone interviewing 

Cedefop European Centre for the Development of Vocational Training 
CGE computable general equilibrium 

DOT Dictionary of occupational titles 

ESJS European skills and jobs survey 

EU European Union 

ICT information and communications technology 

loT internet of things 

ISCED international standard classification of education 

ISCO international standard classification of occupations 

LMSI labour market and skills intelligence 

OECD Organisation for Economic Cooperation and Development 
PIAAC programme for the international assessment of adult competences 
SMT Survey of manufacturing technology 

STAMP skills, technologies and management practices survey 
STEP skills towards employment and productivity 

VET vocational education and training 

WEF World Economic Forum 
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Understanding technological 
change and skill needs 


Skills surveys and skills forecasting 


The world of work is being impacted by a fourth industrial 
revolution, transformed by artificial intelligence and other 
emerging technologies. With forecasts suggesting large 
shares of workers, displaced by automation, in need of 
upskilling/reskilling, the design of active skills policies is 
necessary. 

Conventional methods used to anticipate technological 
change and changing skill needs, such as skill surveys 
and forecasting, have limited scope to provide insights 
into emerging trends. With the increasing use of big data 
and Al methods, analysts have new ‘real-time’ tools at their 
disposal. Skill foresight techniques are also increasingly 
used to gauge in-depth stakeholder information about 
future technologies and skill needs. 

Aseries of short Cedefop guides aims to inform analysts and 
policy-makers about available skills anticipation methods 
used to navigate through the uncertainty of changing 
technologies and skill demands. This first practical guide 
focuses on conventional skills intelligence methods of 
surveys and forecasting. 
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